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ABSTRACT 

Based on Larry Hutchins' analysis of the school 
effectiveness literature, the Profile of School Excellence (PRO-S/E) 
is a diagnostic tool keyed to 11 instructional and organizational 
variables which are positively related to effective/excellent schools 
and which are alterable. The variables are: needs basis; objectives; 
expectations; roles and responsibilities; conditions and resources; 
instructional time and task orientation; assessment use; rewards and 
reinforcement; behavior code; school climate; and parental support 
and involvement. Seven data gathering procedures are used: district 
data forms; school data forms; school rating forms for district 
central office staff; superintendent interviews; principal 
interviews; student questionnaires; and teacher questionnaires. Each 
procedure was pilot tested and revised in school settings. The 
completed PRO-S/E has been administered in eight local education 
agencies (LEA's), ranging in size from 777 to 17,850 students and 
from 2 to 47 schools, in five states. The PRO-S/E provides 
superintendents with diagnostic information through a series of 
graphs (profiles) and a rating report with suggestions for further 
research and development methods for identified problem areas. It is 
especially useful for superintendents of smaller school districts 
with no management information system. Using PRO-S/E, the Appalachia 
Educational Laboratory has enhanced the linkage between v!.e research 
and development community and the practitioner community. (BS) 
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Background and Introduction 

Ihe "school excellence" th?me has attracted considerable national atten- 
tion. Bie report of the National Conunission on Bduational Excellence was the 
first of many major reports which have called attention to the theme. Pro- 
fessional journals have used the excellence theme. Professional associations 
have organized their annual meetings around the theme. The U.S. Secretary of 
Education is sEjonsoring programs to reward "excellent" schools. Most state 
education agencies have organized efforts to address the theme. However, it 
was AEL's observation that the Commission's activity, the journal articles, 
and the convention presentations provided local school leaders few concrete 
ideas about how to improve school effectiveness. Further, AEL suspected that 
the media's coverage of the Commission's report would increase for local 
superintendents the number of inquiries about excellence from board members 
and parents. It was the latter concern, reinforced by comments from many of , 
the local educational agency (LEA) superintendents in AEL's Region, that led 
to the decision to develop the Profile of School Excellence (PRO-S/E)^. 



Hieoretical Framework 



The PRO-S/E is a diagnostic tool keyed to 11 instructional and organiza- 
tional variables research has shown to be positively related to effective 
schools and which are alterable. The purposes of this presentation are to 



As one of the regional educational laboratories sponsored principally 
by the National Institute of Education, AEL serves educators in the states of 
Alabama, Kentucky, Ohio, Pennsylvania, Tennessee, Virginia, and West Virginia. 
However, development of the PRO-S/E was supported only by corporate funds. 
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describe the procedural methodology associated with the project, delineate 
the analytical techniques utilized, ajnd present relevant outcomes. 

The PRO-S/E is based on an analysis of the school effectiveness litera- 
ture adapted from Hutchins (1982) . That analysis. Figure 1, identifies 11 
characteristics positively correlated with effective schools. Ihe authors 
use the terms effective and excellan t synonymously. That is, the authors 
assume that an effective school is an excellent school. For this paper, 
then, an effective/excellent school is one where (1) basic skills achieve- 
ment cannot be predicted from students' socioeconomic status, (2) student 
attendance is regularly above 90%, (3) documented occurrences of vandaVsm 
and delinquency are relatively low, and (4) high satisfaction ratings are . 
given the school by all stakeholder s." The authors have synthesized this 
definition from the literature on effective schools, particularly Edmonds, 
1980; Rutter, 1979; and Goodlad, 1982. 
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Instructional Proce&s Variables 
Studantt Ttachsri Prlncipati 



1. Experiences a clearly identified Accurately diagnose needs; 
need; « 



Superintendents/ 
Central Off ipe Staff 



Develop or select arnl Implement Assess needs for improvement; 
diagnostic, needs sensing systems; 



Champagne, et al, n.d. 



Edmonds 19B0 



Smirk, 1981 



Focuses on a relevant, attainable 
objective; 



Prescribe and communicate 
nature of academic learning 
tasks; 



Define & communicate commun- 
ity-relevant goals & instructional 
objectives (especially In math & 
reading). R utter 1979, Brookover 
and Lezotte 1979, Edmonds 
1979, CA1980 



Plan specific activities that will 
lead to improvement; reject pro* 
grams that are Ineffective; 

Glaser and Ross 1971, Herriott 
and Gross 1979, Rutter 1979 



3. Expects to succeed; 



Communicate the b elief that each 
student can and will succeed; 
Armor et al 1976, Brookover and 
Lezotte 1979. Rutter 1979, 
Edmonds 1979, Glenn 1981, 
California 1980 



Demonstrate a high level of ex- 
pectation for acadmic accomplish- 
ment; 

Wehber 1979, Trisman 1976, 
Venezky andWinfield 1979 



Set high expectations for changes 
that will further improve prac- 
tice; 

Brookover and Lezotte 1979, 
Austin 1978 



4. Assumes an appropriate level of Prepare students to assume 
responsibility for learning, coop* responsibility for learning; 
erates with others, and participates 
in a broad range of academic and 
non-academic activities; ^^^^^ ^g^g 



Encourage (and receive) responsi- 
ble autonomy at the classroom 
and building level; 

Armor 1976. Hargrove et al 1981, 
Levine and Stark 1981 



Maintain sufficient, flexible, com- 
plex, known organization model; 
Establish coordinated but indepen- 
dent Improvement efforts at all 
levels in the system. Salganic 1980,\ 
Brookover 8i Lezotte 79, Austin 78 



5. Receives appropriate academic 
and skill experiences in an inter- 
active, participatory mode until 
he/she is successful; 



Group students appropriately; pre- 
^sent and model Information and 
skills In an Interactive way to mo- 
tivate all students; NY 1974, Rut- 
ter 1979, Rosenshine & Stevens 
1981, Stallings 1980 



Obtain and provide alknecessary ' 
materials and technical resources; 



Provide inservice training, personal 
developm'ent opportunities, and 
necessary Information; 



Levine and Stark 1981, NIE 1978 California 1980, Armor 1976 



6. Attends to and engages In academ- 
ic tasks; 



7. Has knowledge of outcomes 
(success); 



Proves adequate time fdr discov- 
ery, practice, and transfer of 
learning; 



Fisher e| al 1980, Evertson 1980 



Establish and uniformly enforce 
attendai^ce policies and time 
assigned to academic tasks; 

Harnischfeger and Wiley 1978, 
NIE 1978 



Create time for communication, 
learning, training, etc.; 



Little 1981 



Monit rs and evaluates progress 
and rwporu (feedback) results; 

WebL^r 1971, Rutter 1979, R> 
senshlne 1980, Fisher etal 1980 



Design and operates an evaluation 
system that reports a continuous 
flow of nationally-normed, locally 
relevant, criterion-referenced In- 
formation. Edmonds 1979, Web- 
ber 1979, Levine and Stark 1981 Fullan 1980 



Monitor, evaluate, and report 
progress; 



8. Experiences a sense of accomplish* Rewards and/or praises approprl- 
ment and achievement; ately all those deserving; 



Rutter 1979 



Organize and carry out proce- 
dures for recognizing accomplish- 
ment and achievement by stu* 
donts and all^aff; 
r^lE 1978, Bredo 1977, Peterson 
1980 



Acknowledge and reward efforts 
at improvement, reduce risk of 
failure; 



Little 1981 



9. Has access to, accepts, and coop- 
erates In creating a predictable, 
pleasant, safe, clean environment; 



Establishes and fairly enforces 
rules, structure, routines, and con* 
sequences; 

Rosenshine and Stevens 1981, 
Edmonds 1979, Rutter 1979, 
Stallinos 1981. Evertson 1980 



Collaboratively and fairly tnforcv 
standird operating pi'ooadurt (rultt. 
oodti, etcJ that minimize burdtmoma 
tMkt and providt a pradlctablt, plaa-^ 
lant, safe, dean anvironment: Wabbsr 
1971, NIE 1978, Rutttr 1979, Levine 
and stark 1991 



Establish routines and strategies 
for improvement (e.g., research); 



Little 1981. Hall 1979. 1980 



Receives from and returns to those 
around him/her a sense of caring, 
personal concern, interest, respect, 
commitment and support for per* 
sons, property, and ideas; 



Receives from 8i returns to those 
around a sense of caring, personal 
ooncerni interest, respect, commit- 
ment 8( support tor persons, pro- 
perty, and Ideas; Good & Grouws 
197;^, Webber 1971, Rutter 1979 



Create and model a colleglal en- 
vironment in which individuals 
and ideas are respected and 
supported; Webber 1971, Rutter 
1979, NIE 1978 



Plan collaborative activities that 
are sensitive to concerns and 
differences in group situations; 

Little 1981, Hall 1979, 1980, 
Hershey and Blanchard 1977 



Receives and accepts parental 
and community interest. Involve- 
ment, help, and approval for 
academic learning. 



Actively encourages and solicits 
parental Interest, participation, 
and support for learning. 



Armor et al 1976 



Establish procedures that encour- 
age meaningful parental and com- 
munity Interest involvement and 
support. 

Brookover and Lezotte 1979 



Involve parents, community mem- 
bers, students, and staff appropri- 
ately In improvement planning, 
activities, and evaluation: CA 1980. 
Barman & McLaughlin 107B, 1977, 
1978, Little 1981, Glenn 1981 



^oted from Larry Hutchlns, et al., 19S2. 
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Figure 1 



Definitions for the 11 variables used in the PRO-S/E follow: 



1. NEEDS BASIS: The degree to which school personnel use an in-place system 
for identifying, teaching, evaluating, and remediating students' learn- 
ing needs. 

2. OBJECTIVES: The degree to which school personnel prescribe and canmuni- 
cate to students relevant and attainable objectives for each academic 
course. 

3. EXPECTATIONS: tftie degree to which school personnel communicate clearly 
to each student, the belief (the expectation) that each can and will 
succeed in attaining prescribed academic objectives. 

4. ROLES 6 RESPOHSXBILITIES: Ohe degree to which school personnel prepare 
each student to assume an appropriate level of responsibility for learn- 
ing, to recognise and cooperate with others playing key roles in the 
school system, and to participate in a broad range of academic and 
non-academic activities. 

5. CONDITIONS 6 RESOURCES: The degree to which school personnel provide 
students exemplary conditions of learning, that is, they group students 
appropriately; they present and model information and skills in an intor- 
active way that properly motivates students; and they use excellent in- 
structional materials to assure maximum student participation and 
success. 

6. INSTRUCTIONAL TIME 6 TASK ORIENTATION: The degree to which school 
personnel provide students maximum instructional time during class 
periods and assure the students attend to and engage in academic tasks 
during class time. 

7. USE OP ASSESSMENT: The degree to which school personnel use assessment 
data as the basis for informing students of their academic progress and 
informing teachers of their students* remediation needs. 

8. REWARDS 6 REINF0RCQ4ENT: The degree to which school personnel use an in- 
place system of reinforcement that recognizes the accomplishments and 
achievements of students and staff. 

9. CODE OF BEHAVIOR: The degree to which school personnel communicate 
clearly and enforce equitably rules, structure, routines, and conse- 
quences governing student behavior. 

10. SCHOOL CLIMATE?\The degree to which school personnel create and model a 
collegial environihent in which students receive and return to those 
around them a sense of caring, personal concern, interest, respect, 
commitment, and support for persons, property, and ideas. 

11. PARENTAL SUPPORT ft INVOLVEMENT: Ohe degree to which school personnel 
have established procedures that encourage meaningful parental and 
community interest, involvement, and support in students' academic 
learning. 
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Development Process 

To assess the strength of the 11 characteristics (variables) , the PRO-S/E 
utilizes 7 data gathering procedures: district data form, school data form, 
school rating form for district central office staff, superintendent interview, 
principal interview, student questionnaire and teacher questionnaire. 

Each procedure was pilot tested in school settings, ^e data forms, rat- 
ing forms and interviews were revised until trial use indicated they were in 
satisfactory form. Special attention was given to the two questionnaires. 
Each questionnaire included at least 5 items to measure each of the 11 
variables. Utilizing the Dale-Chall and Prye techniques, the reading levels 
of the questionnaires were determined to be appropriate for the intended 
audience: 6th-grade for students and 13th-grade cfor teachers. These question- 
naires were used to assess the perceptions of students and teachers from six 
high schools in the pilot test site. OJie perceptions of students and of 
teachers were compared with the perceptions of appropriate central office 
staff to assess the discrimination power of the questionnaires. District 
central office staff members' ratings were provided on the school rating 
form. fHie form consisted of the 11 definitions and a ten-point rating scale. 
A coefficient of concordance (w) was computed for each school across six 
raters. For each school the <w) > .90 and was statistically significant 
(p <.01), The district central office staff's ratings of the schools and the 
teachers' and students' questionnaire ratings of the school were significantly 
correlated (p <.05). Thus, the questionnaires were judged to be satisfactory 
measures of subjects' perceptions of the strength of the 11 PRO-S/E variables. 
Since the PRO-S/E was intended only to gather perceptual data for diagnostic 
use, no other developmental testing of the procedures was conducted. 
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Implementation Process 

0 

'Oie completed PRO-S/E has now been administered in eight LEA's across 
five states, ^e LEA's range in size from 777 to 17»850 students and from 
2 to 47 schools.. At these sites AEL used seven PRO-S/E data sources: 

1. Members of the district central office staff provided ratings 
of the schools to be studied on the 11 characteristics. 

2. A 50 percent sample of the district's non>itinerant teachers 
responded to the PRO-S/E Teacher Questionnaire. 

3. A 10 percent sample of the district's 6th-12th grade students 
responded to the PRO-S/E Student Questionnaire. 

4. The principals completed a PRO-S/E Data Form that provided general 
" information about their respective schools. 

5. The superintendent completed a PRO-S/E Data Form that provided 
general information about the school system. 

6. Each principal was interviewed by one of the researchers who used a 
PRO-S/E Principal Interview Form. 

7. The superintendent was interviewed by the team of researchers who 
used a PRO-S/E Superintendent Interview Form. 

Ratings of the schools by the central office staff (#1) and question- 
naire responses by teachers (12) and by students («3) were weighted and then 
plotted on PRO-S/E Profile Charts. The district and school data forms (#4 
and #5) and the interviews with the principals and the superintendent (#6 and 
«7) were used in the interpretation and discussion of the profiles. 
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Results 

Hhe PSO-S/E provides superintendents with diagnostic Information. It 
provides a series of graphs (profiles) and a brief, understandable narrative 
report which show and tell how a school system rates on factors research has 
shown to be associated with effective schools, OSie graphs, see Figure 2] show 
visually whether a school Is perceived as "superior" (above 901), "acceptable" 
(between 90% and 70%) , "marginal" (between 70% and 50%) , or "unacceptable" 
(below 50%) on these factors. These rating categorlt^s, established by the 
authors as a result of the pilot test, have worked well In subsequent Implemen- 
tations. The report contains also a comparison of the district's PRO-S/E data 
with the literature's four-part definition of an effective school. Finally, 
the report suggests R & D-based methods for developing empirical data to 
further assess variables perceived to be problematic. - > 

PRO-S/B reports are front 40 to 100-plus pages depending on the size of the 
client-district and the number of schools the client elects to include in lli^ 
study. The report begins with one to three pages defining the purpose and 
limitations of the diagnostic study and the procedures followed. Next, school- 
by-school findings are presented alphabetically. The profile graph for each 
school is followed by 2-3 pages o* narrative. Finally a 10-15 page confiden- 
tial raeraorandura to the superintendent discusses perceptions across the schools 
and suggests alternative, jR & D-based courses of action for dealing with 
problem areas. 

The PRO-S/E Increases the data base superintendents have to use in making 
decisions about what to do next to improve the effectiveness of their respec- 
tiv..^ LEAS. It is, therefore, highly valued by those superintendents who have 
used it. AEL recognizes and advertises the limitations of the PRO-S/E~it is 
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diagnostic, percept ion -based, and is based on limited input from some LEA stake- 
holders. However, the process is relatively low-cost » non-obstrusive and 
quick— final report back to the district within six weeks of the researchers' 
site visit. It seems, therefore, to have considerable utility to super in- 
tendents of smaller school distriq^tjB who have no managementVinformation system 
in place and who want a rational^ basis for proi^osing particular school excel- 
lence initiatives to their respective school boards or for deciding how to deal 
with competing school improvement priorities. 

Growth in R & D-based school improvement activities during the last 5-10 
years has been noteworthy; Using the effective schools research as a framework 
for school improvement activities is a good, example of how research can contrib 
ute to educational practice. AEL is designed to optimize the linkage between 
the R & D community and the practitioner community. Through* the use of the 
PRO-S/E, AEL has enhanced this linienge; through linlcages with other technical 
assistance providers, regional educational laboratories, and R 6 D centers, AEL 
can identify also research-based processes and products that districts can use. 
to address areas diagnosed, as needing further attention. 
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